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1 .  BACKGROUND 


There  is  considerable  evidence  that  the  basic  functional  unit  for  higher-level 
processing  in  the  cortex  is  the  netlet  or  neuronal  assembly/(pool  or  group)  [1]-[10].  This 
includes  extensive  analytical  and  modeling  work  of  netlets  carried  out  independently  by 
several  groups.  Nearly  all  this  work  points  to  the  possibility  that  netlet  dynamics,  namely 
time,  can  be  described  by  the  discrete  time  eyolution  of  the  actiyity  A(n) 
which  IS  the  percentage  of  neurons  actiye  at  any  instant  of  time.  Plots  of  A(n+1)  ys  A(n) 
obtmned  under  a  range  of  circumst^ces  and  assumptions  are  found  to  inyariably  resemble 
a  distorted  version  of  the  quadratic  or  logistic  map.  The  Logistic  map  is  a  nonlinear 
iterative  map  on  the  unit  interval  that  exhibits  complex  orbits  depending  on  the  value  of 
nonlinearity  (control  or  bifurcation)  parameter  of  the  map  [1 1].  The  similarity  between  the 
u  u  A(n-t-l)  vs.  A(n)  and  that  of  the  logistic  map  has  also  been  noted  by 

Harth  [10]  who  also  mentions  that  complex  and  unpredictable  sequences  A(n)  were 
observed  in  some  of  their  early  simulations  of  netlets  suggesting  that  certain  regions  of  the 
netl^  s  parameter  space  may  have  led  to  observation  of  chaos  in  addition  to  the  periodic 
and  fixed  point  modalities  they  usually  observed. 

In  light  of  this  evidence  we  have  conjectured  that  cortical  networks  can  be  modeled 
and  numerically  studied  in  an  efficient  way  by  means  of  coupled  populations  of  logistic 
processing  elements  [12],  To  test  this  conjecture  we  have  studied  the  dynamics  of  such  a 
network  when  it  is  subjected  to  external  stimulus  patterns  that  change  in  time.  The 
networks  we  study  differ  from  Kaneko’s  coupled  map  lattices  (CMLs),  [13]-[14],  in 
severe  ways:  (a)  The  networks  described  here  employ  parametric  rather  than  the  diffuse 
coupling  used  m  CMLs,  (b)  The  coupling  is  nonlinear  representing  the  possibility  that  the 
interaction  between  netlets  can  depend  on  the  activity  of  the  netlets  and  on  the  number  of 
connecting  one  netlet  to  another,  (c)  Our  parametrically  coupled  logistic  nets 
(PCLNs)  can  be  externally  driven  by  time  varying  or  stationary  patterns,  or  by  composite 
patterns  that  are  partially  time  varying  and  partially  stationary,  (d)  In  the  PCLN,  control 
over  network  dynamics  is  gradually  handed  over  from  initially  entirely  extrinsic  control  to 
eventually  entirely  intrinsic  control.  This  handing  over  of  control  over  network  dynamics 
from  extrinsic  to  intrinsic  is  biologically  plausible  and  is  inspired  by  the  remarkable 
biophysical  observation  made  by  Freeman  and  coworkers  [15]  regarding  gradual 
disappearance  of  the  trace  of  a  sensory  stimulus  applied  to  the  olfactory  bulb  of  rabbit  as  it 
was  followed  deeper  in  the  sensory  cortex  where  it  was  found  to  eventually  vanish  in  a  sea 
of  intrinsically  dominated  activity.  Similar  behavior  has  apparently  been  observed  by 
Freeman’s  group  in  other  sensory  modalities. 

2.  PRESENT  RESEARCH 

The  preceding  remarks  suggest  that  networks  of  parametrically  coupled  logistic  maps 
are  biomorphic  in  the  sense  that  they  offer  an  efficient  way  to  study  the  functional 
complexity  of  cortical  networks  in  order  to  understand  the  way  they  perform  higher-level 
functions.  Such  higher-level  functions  are  beyond  the  capabilities  of  present  day  sigmoidal 
networks,  and  incorporating  them  in  ^ficial  network  offers  a  way  for  increasing  then- 
processing  power  and  for  widening  their  scope  of  application. 

One  objective  of  our  research  program  in  biomorphic  neural  networks  and  then- 
applications  has  been  to  develop  networks  that:  (a)  possess  biomorphic  processing 
elements  whose  operation  mimics  that  of  the  basic  functional  unit  in  the  brain  (cortex),  (b) 
are  naturally  suited  for  receiving  and  processing  spatio-temporal  inputs  (time  vaiying 
patterns)  and  can  be  trained  to  classify,  recognize,  or  generate  such  patterns,  (c)  are  able  to 
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compute  with  diverse  state-space  attractors  that  include  static  (fixed-point)  and  dynamic 
attractors  with  the  latter  including  period-m,  and  chaotic  attractors.  This  last  requirement  is 
motivated  by  the  observation  that  the  brain  is  essentially  a  high-dimensional  nonlinear 
dynarmcal  system  that  can  exhibit,  in  principles  all  three  types  of  attractors,  fixed-point 
priodic  and  chaotic,  in  its  operation,  (d)  ability  to  undergo  bifurcation  (sudden  switching) 
between  state-space  attractors.  Such  bifurcation  can  be  a  source  for  versatility  and  possible 
through  chaotic  search  for  novel  solutions  in  the  state-space  of  the 


Most  sensory  information  we  deal  with  in  daily  life  is  seldom  static  but  is  usually 
dynamic  producing  patterns  of  sensory  activity  that  are  perpetually  changing  in  time.  This 
occurs  in  all  sensory  modalities:  seeing,  hearing,  olfaction  (smell),  and  touch.  Similarly 
most  signals  produced  by  the  brain  to  control  motor  function  are  spatio-temporal  in  nature. 
A  similar  situation  is  expected  to  apply  in  advanced  artificial  neural  networks  intended  for 
higher-level  processing.  Such  network  are  expected  to  find  application  in  the  processing  of 
spatio-temporal  signals  of  the  kind  encountered  for  example  in  radar,  sonar,  and  speech. 
They  are  also  expected  to  be  able  to  generate  input  specific  spatio-temporal  signals  in 
response  to  specific  static  or  dynamic  input  (stimulus)  patterns  which  is  desirable  for 
robotic  systems  especially  when  hardware  compactness  and  power  efficiency  are  required. 
This  will  be  analogous  to  central  pattern  generation  in  the  nervous  system  needed  to  control 
movement  in  living  things. 

Particularly  interesting  to  us  are  networks  that  can  accept  spatio-temporal  signals  like 
those  produced  in  remote  sensing  and  recognition  tasks  as  in  radar  and  sonar.  Success  in 
this  arena  can  be  useful  in  other  areas  and  can  lead  to  systems  that  could  recognize 
continuous  speech,  or  could  directly  convert  spoken  commands  into  complex  spatio- 
temporal  control  functions.  For  this  reason  we  have  been  studying  the  behavior  of 
parametrically  coupled  networks  of  bifurcation  processing  elements  or  cells.  The  focus  has 
been  on  parametrically  coupled  networks  of  logistic  maps,  for  the  reasons  given  earlier  but 
other  Iterative  maps  on  the  interval  like  the  sine-circle  (standard)  map  can  also  be  used. 

This  report  is  meant  to  summarize  and  present  a  number  of  recent  novel  results  that 
have  inaportant  implications.  They  were  obtained  during  the  past  quarter  in  numerical 
simulation  of  PCLNs  under  external  stimulus. 

The  network  studied  is  shown  in  Fig.  1.  It  consists  of  N  parametrically  coupled 
logistic  maps  arranged  on  a  line  (one  dimensional  geometry)  to  simplify  the  display  of  their 
time  evaluation  as  will  be  illustrated  below.  Parametric  coupling  means  the  nonlinearity, 
(control  or  bifurcation  parameter  jii  of  the  i-th  map  is  modulated.  In  the  network  /tj  is 
modulated  by  both  extrinsic  and  intrinsic  influences  according  to 

i+Nj 

Z  gij(n),  . i  =  1,2,...N  (1) 

j=i-Ni 

G(n)  =  exp(-an) 

In  eq.  (1)  the  first  term  represents  the  extrinsic  (sensory)  input  to  the  i-th  logistic 
processing  element  or  cell  and  the  second  term  represents  the  net  intrinsic  input  to  the  i-th 
cell  through  feedback  from  all  other  cells  connected  to  it.  In  eq.  (1)  n  is  discrete  integer 
time,  2Ni  is  the  number  of  logistic  cells  connected  to  the  i-th  cell  i.e.  the  number  of  cells 
falling  within  the  “connection  radius  R^”  which  is  taken  to  be  the  same  for  all  cells. 


W(n)  =  e(n)g?(n)  + 

.ZIN  j 
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e{n)=^  exp(-  an) , 

where  is  in  [0,l]  and  a  >  0  is  positive  real  number. 

and  Wy  are  nonlinear  activity  dependent  coupling  functions  with  two  possible  forms  each: 

\^Bf  (m, )  =  quantized  version  of  gf  [u^ )  (//*  levels) 

[5^  (x^ )  =  quantized  version  of  g,j  (x^ )  levels) 


Fig.  1.  Parametrically  coupled  logistic  network  (PCLN)  consisting  of  N  bifurcation  (logistic)  processing 
elements  or  cells.  The  network  employs  (a)  novel  biologically  plausible  nonlinear  (activity) 
dependent  coupling  functions  between  cells  each  representing  a  netlet  and  (b)  a  biologically 
plausible  gradual  transfer  of  control  over  network  dynamics  from  initially  totally  extrinsic 
(sensory)  control  to  totally  intrinsic  control.  The  network  can  be  driven  externally  by  spatio- 
temporal  inputs  provided  by  the  stimulus  generating  network  that  employs  an  array  of  uncoupled 
logistic  maps  to  conveniently  produce  a  variety  of  static,  time-periodic,  chaotic,  or  composite 
signals  made  of  any  mix  of  these  three-types  of  signals.  The  quantized  versions  of  the  coupling 
functions  are  useful  for  studying  the  coarse-gain  dynamics  of  the  PCLN. 
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gi(n)  =  4(ui(n))'^i  is  the  extrinsic  (sensory)  input  to  the  i-th  cell  with  variable 
Ui(n)G  [0,1]  being  produced  in  the  simulation  for  convenience  by  a  sensory  logistic  map 
according  to:  ui(n+l)  =  ui(n)  (l  -  ui(n))  with  ui(0)  =  0.5  and  being  a  fixed 

control  parameter  of  the  i-th  stimulus  generating  logistic  map.  Selecting  in  [0,4] 

enables  the  production  of  a  wide  range  of  stationary,  periodic  (period-m)  or  chaotic  patterns 
u  j  (n)  or  any  desired  combination  of  such  patterns  on  i  depending  on  the  values  one  selects 

for  jU?.  Thus  by  changing  the  control  vector  of  the  N  stimulus  generating  logistic 
cells,  a  wide  range  of  spatio-temporal  driving  signals  can  be  applied  to  the  network.  The 
coupling  factor  wj  ranges  between  0  and  o®.  For  example  wj  =  0  produces  g?(n)  =  4 
which  means  the  extrinsic  contribution  tends  to  make  (n)  high  with  the  result  that  the  i-th 
processing  cell  would  tend  to  be  chaotic.  On  the  other  hand  wj  =  oo  yields  g?(n)  =  0, 

because  the  state  variable  uj  of  the  logistic  map  is  in  [0,1].  This  means  that  small  values 
of  Wj  introduce  disorder  while  larger  values  introduces  inhibition.  Similarly,  the  quantity 
Sij(^)  (1)  represents  the  input  from  the  j-th  cell  to  the  i-th  ceif  it  has  a  form  similar  to 

gl  (n),  namely  g..(n)  =  4[Xj(n)]  ^  with  Cij  being  in  [0,~]  and  Xj(n)  is  the  state 
variable  of  the  j-th  logistic  processing  cell  of  the  network  governed  by: 
Xj(n-H)  =  /tj(n)  (l  -  Xj(n))  where  /ti(n)  is  given  by  eq.  (1)  and  Xj(n)  is  also  in 

[0,1].  Note  the  nonlinear  dependence  of  g-(n)  on  Uj(n)and  gjj(n)  on  Xj(n)  serves  two 

purposes.  One,  it  confines  their  combined  contribution  to  jUj(n)  to  the  allowable  range 
[0,4],  and  second,  changing  the  values  of  wj  and  Cjj  provides  control  over  the  level  of 

excitation/disorder  or  inhibition  injected  into  the  dynamics  of  the  i-th  cell,  and  hence  into 
the  network  as  a  whole,  by  the  i-th  sensory  ceU  or  by  the  j-th  processing  cell  respectively. 

The  par^eter  a  in  eq.  (1)  is  a  positive  real  constants  whose  value  determines  the  speed 
with  which  control  over  the  dynamic  of  the  network  is  handed  over  from  initially  entirely 
control  extrinsic  to  eventually  and  entirely  intrinsic  control.  A  value  of  G  ^  =  1  means  that 
initially  the  dynamics  of  the  network  are  totally  controlled  by  the  exrtnsic  (sensory) 
pattern.  The  passing  of  control  over  network  dynamic  from  initially  entirely  extrinsic  to 
eventually  entirely  intrinsic  is  meant  to  reflect  recent  intriguing  biophysical  finding  by 
Freeman  and  coworkers  concerning  the  fading  and  eventual  disappearance  of  the  traces  of  a 
sensory  stimulus  as  it  propagates  deeper  into  the  sensory  cortex  [1].  Similarly  the 
nonlinear  coupling  function  gjj(Xj(n))  between  cells  is  biologically  inspired  as  nonlinear 

activity  dependent  coupling  between  netlets  is  more  plausible  than  linear  coupling.  Indeed 
simulation  results  indicate  that  nonlinear  coupling  is  essential  and  has  several  advantages 
over  linear  coupling  which  is  expressed  by 


W(n) 


min  < 


G(n)WiUi(n) 


l-g(n) 

2Ni 


i-t-Nj 

I  WijXj(n), 

j=i-Ni 


4 


i  =  l,2,...N 


(2) 


We  find  that  when  the  ranges  of  wj  and  wjj  are  confined  to  values  that  do  not  violate  the 
requirement  of  the  logistic  maps  namely  that  they  fall  in  the  interval  [0,4]  (otherwise 
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can  have  values  exceeding  4  which  is  not  allowed  by  definition  of  the  logistic  map),  the 
linear  coupling  does  not  work  in  the  sense  that  the  network  is  extinguished:  Xj  (n)  ->  0 . 

The  PCLN  represented  by  eqs.  (1)  and  (2)  is  clearly  an  example  of  an  extremely 
complex  system.  The  general  problem  in  studying  complex  systems  is  selecting  or  finding 
the  regions  in  the  parameter  space  of  the  systems  for  which  meaningful  and  hence 
potentially  useful  behavior  can  be  observed.  In  our  case,  meaningful  behavior  could  be  for 
instance  behavior  similar  to  that  seen  in  the  brain  with  modem  imaging  techniques  primarily 
with  functional  magnetic  resonance  imaging  (fMRI)  and  positron  emission  tomography 
(PET)  scans.  Especially  meaningful  would  be  the  emergence  in  our  network  of 
corticomorphic  behavior  only  when  the  selection  of  network  parameters  is  guided  by  broad 
biological  principles  of  cortical  operation  and  organization.  Indeed  this  is  the  case  for  the 
PCLNs  we  have  studied  and  this  makes  the  results  quite  exciting  with  potential  to  lead  to  a 
new  generation  of  powerful  neural  networks  namely  dynamical  networks  that  compute  with 
diverse  attractors. 

Before  we  go  on  to  describe  the  PCLN  and  give  examples  of  its  behavior  it  is  worth 
listing  the  specific  desirable  properties  we  seek  in  a  dynamical  network.  These  include: 

•  The  spatio-temporal  response  of  the  network  should  be  stimulus  specific  i.e.  the 
response  of  the  network  should  be  able  to  differentiate  between  different  spatio- 
temporal  input  patterns;  in  brief  we  wish  the  response  to  be  stimulus  specific. 

•  Rapid  convergence  to  a  steady  state  pattern. 

•  Independence  of  the  response  from  the  initial  state  vector  X(0)  of  the  network 

i.e.,  from  Xi(0)  e  [0,1],  i=l,2,...N.  Note  the  initial  state  UjfO)  of  the  stimulus 
generating  logistic  cells  is  fixed  to  allow  repeatability  and  coherence  of  the 
spatio-temporal  input  patterns  used  (in  the  simulation  below  ui(0)  is  arbitrarily 
fixed  at  0.5. 

•  Network  response  should  depend  on  the  coupling  coefficients  matrix  Cy  i,j  = 
1,2,...N.  Recall  Cy  sets  the  form  of  the  activity  dependent  nonlinear  coupling 

function  gy ( Xj  )=  4[  Xj  (n)]  U  between  the  j-th  and  i-th  processing  cell  of  the 
network  offers  a  mechanism  for  adaptation  and  learning  in  the  network. 

•  Consistency:  small  changes  in  the  input  stimulus,  brought  about  by  small 

changes  in  the  stimulus  generating  vector  ,  should  not  change  the  response, 
i.e.,  many  similar  input  patterns  produce  the  same  response  or  negligible 
changes  in  the  response.  When  a  learning  algorithm  is  developed,  consistency 
might  be  expanded  to  include  generalization. 

•  In  the  absence  of  extrinsic  (input)  stimulus  (undriven  net),  the  response  of  the 
network  should  depend  on  the  initial  state  of  the  network  in  a  manner  analogous 
to  that  in  conventional  recurrent  networks  (Hopfield-type  network)  that  compute 
with  static  (point)  attractors.  The  behavior  here  would  be  similar  but  with 
dynamic  in  addition  to  static  attractors  occurring.  The  dependence  on  initial 
conditions  should  exhibit  basins  of  attraction  just  like  conventional  point 
attractor  network. 
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•  The  network  should  enable  testing  scenarios  for  adaptation  and  learning, 
especially  those  that  are  holography-like  in  nature  because  these  may  offer  rapid 
“one-shot”  learning. 

•  The  behavior  should  resemble  known  attributes  of  cortical  nets  that  is  being 
uncovered  by  functional  imaging  methods  like  PET  and  fMRI. 

As  will  be  demonstrated  below,  we  find  that  there  is  indeed  a  range  of  biologically 
plausible  network  attributes  and  parameters  for  which  the  network  in  Fig.  1  exhibits  the 
above  desirable  properties.  One  such  attribute  is  the  use  of  local  instead  of  the  global 
connectivity  used  in  our  earlier  work.  In  the  present  network  a  cell  (netlets)  is  connected  to 
its  near  neighbors  that  fall  within  a  connection  radius  of  €  [a,b]  such  that  the  number  of 
cells  N[  within  Rc  is  considerably  lower  than  the  size  N  of  the  network.  Although  pools 
of  processing  cells  (netlets)  in  the  cortex  do  receive  and  send  long  range  fibers  to 
communicate  with  other  distant  pools,  the  processing  by  netlet  pools  is  dominated  by  local 
interactions.  In  earlier  simulations  that  employed  global  connectivity  the  network  was 
found  to  exhibit  clustering  into  a  number  of  phase-locked  groups  with  the  cells  within  a 
cluster  being  synchronized.  In  a  network  of  N  »  Nj  cells,  we  find  now  that  by  setting  a 
=  o,  b  =  1  or  2  and  selecting  the  value  of  Rc  e  [o,b]  for  any  cell  randomly,  the  network 
can  be  made  to  exhibit  isolated  clustering  i.e.,  small  size  clusters  of  active  cells  separated 
by  regions  of  silent  ones.  This,  as  was  mentioned  earlier,  is  reminiscent  to  the  images  of 
brain  activity  observed  in  fMRI  and  PET.  Cells  within  an  isolated  cluster  are  found  to 
possess  a  variety  of  types  of  orbits,  i.e.,  fixed  point  period-m,  and  intermittent,  quasi- 
periodic  and  even  chaotic  that  depend  on  the  input  stimulus  and  the  range  of  nonlinear 

coupling  function  gij(Xjj  used.  This  is  considerably  richer  behavior  than  the  identical 

period-m  synchronized  orbits  within  a  cluster  observed  in  earlier  work.  Fix-point,  period- 
m  and  chaotic  orbits  can  coexist  within  isolated  clusters  and  which  type  of  these  orbits  is 
more  prevalent  can  be  controlled  by  selecting  the  range  of  the  coefficient  Cjj  of  the 

nonlinear  coupling  functions  gij(xj(n))  =  4  (Xj(n))^y .  The  simulation  results 

presented  below  were  obtained  for  random  coupling  functions  formed  by  selecting  Cjj 

randomly  with  equal  probability  in  the  range  [A,B]  i.e.,  Cjj  e[A,B].  Decreasing  A 

towards  zero  in^es  the  increases  the  level  of  excitation  and  leads  to  a  tendency  of  the 
network  to  exhibit  chaotic  orbits  within  some  of  the  isolated  clusters.  Increasing  the  value 
of  B,  on  the  other  hand,  to  values  above  1  tend  to  introduce  inhibition  in  the  network  by 
making  the  cell  orbits  within  the  isolated  clusters  more  ordered  i.e.,  of  the  period-m  or 
fixed-point  variety  and  decreases  at  the  same  time  the  size  and  number  of  active  clusters  and 
further  increase  in  B  can  cause  all  activity  in  the  network  to  be  suppressed.  The  parameters 
A  and  B  can  thus  be  used  to  control  the  level  of  excitation  and  disorder  in  the  network 
(through  A)  or  the  level  of  inhibition  (through  B). 

At  the  present,  the  time  resolution  of  fMRI  and  PET  is  not  sufficient  to  resolve 
temporal  activity  within  the  active  clusters.  It  would  naturally  be  very  meaningful  if  future 
imaging  technology  would  be  able  to  resolve  the  spatio-temporal  and  not  only  the  spatial 
brain  activity  observed  under  sensory  stimulus. 

It  is  worth  noting  that  clustering  into  a  small  number  of  small  isolated  pockets  of 
activity  can  be  desirable  and  beneficial  in  the  study  of  adaptation  and  learning  algorithms 
based  on  mutual  information  because  the  sparsity  of  the  resulting  mutual  information  matrix 
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and  the  greatly  reduced  computational  effort  needed  to  generate  it  from  the  orbits  of  the 
active  cells  in  the  network. 

The  initial  state  vector  X(0)  of  the  processing  elements  was  selected  randomly  i  e 
Xi(0)  i=l,2,...N  was  selected  randomly  with  equal  probability  in  [0,1].  This  is  done  to 
ensure  agreement  with  cortical  networks  where  the  processing  units  can  be  in  any  state  at 
the  instant  the  cortical  network  receives  the  effect  of  a  sensory  stimulus  or,  an  input  from 
other  cortical  networks.  The  initial  states  ui(0)  i=l,2,...N  of  the  cells  in  the  stimulus 
generating  network  were  fixed  arbitrarily  as  mentioned  earher  at  0.5  in  order  to  preserve 
repeatability  of  the  stimulus  patterns  generated. 

A  program  in  Windows  was  developed  to  exercise  the  network  of  Fig.  1  under  a 
wide  range  of  conditions  and  parameters  to  study  its  complex  behavior.  The  following 
parameters  were  defined.  ^ 


N  —  number  of  processing  cells.  Also  equal  to  the  number  of  stimulus 
generating  cells. 

Rc  —  connection  radius  selected  in  the  range  a,b  i.e.,  e[a,b]  when  a,b  >  0 
defines  the  number  of  neighboring  cells  a  given  cell  is  connected  to. 

Self-connection  —  Selects  whether  a  cell  communicates  with  itself  or  not.  If  the 
processing  unit  in  the  cortex  is  a  netlet,  then  because  a  netlet  is  composed 
of  a  large  number  of  interacting  neurons,  [l]-[7],  self-connection  is  there. 
Indeed  we  find  that  the  interesting  behavior  of  the  PCLN  described  below 
occurs  when  the  self-connection  feature  in  the  program  is  selected. 

Periodic  Boundary  Condition  —  Selects  whether  a  circular  or  line  array  of  cells 
geometry  is  used  or  not.  This  attribute  was  found  to  have  little  effect  on 
the  behavior  of  the  network  described  below  because  of  the  small 
connection  radius  used. 


^’^0  parameters  defining  the  speed  with  which  control  over  dynamics  is  passed 
from  extrinsic  to  intrinsic  control,  (see  eq.  (1)). 

^i^  ’  ^ij  selected  randomly  in  the  range 

[A,B].  For  the  results  in  Fig.  2  were  obtained  with  Cjj  e  [0-3]  in 
order  to  demonstrate  chaotic  orbits  coexisting  with  other  types  of  orbits. 

Xj  (0)  :  Initial  state  of  the  processing  cells  selected  randomly  in  [0,1]. 

Stimulus  :  Selects  a  number  of  distinct  shapes  for  the  stimulus  generating  vector 


Examples  of  the  behavior  of  the  network  are  given  in  Fig.  2  for  a  network  of  N=100 
cells  (netlets).  The  cell  index  is  i=0,l,...99  arranged  vertically.  The  left  most  panel  given 

the  control  or  nonlinearity  vector  }j.^  of  the  stimulus  generating  network.  The  values  of 
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Example  of  the  behavior  of  the  driven  parametrically  coupled  logistic  net  of  N=100  cells  Cells 
are  numbered  vertically  from  i=0  to  99.  Each  cell  has  self-connection  and  is  connected  to  its 

adjacent  cells  with  probability  of-.  The  stimulus  generating  vector  Jl^  is  shown  in  the  left 

panel,  the  spaUo-temporal  pattern  Ui(n)  produced  by  the  i-th  stimulus  generating  logistic 
element  IS  shown  in  the  next  panel  for  the  n=90  to  n=99  iterations.  The  central  large  panel 
shows  the  temporal  evolution  X(n)  =  {Xi(n)  ,  i  =  0,1,...  N  =  99}  for  the  first 


Figure  2  caption  fcont’d) 


hundred  iterations  (n  =  0,1, 2, ...99).  The  initial  states  Xi(0)  for  iterating  the  logistic 
processing  cells  was  selected  randomly  in  [0,1]  and  the  control  parameter  /ii(n)  was  that  of  eq. 
(1)  of  the  text.  Note  the  rapid  emergence  of  clusters  and  convergence  into  ‘‘steady  state'’  i.e,, 
persistent  types  of  orbits:  for  example  period-4  orbit  for  i=3  ,  and  i=83,  and  chaotic  orbit  for 
i=56  and  91.  Cells  with  fixed  point  orbits  encoded  by  uniform  color  are  easily  identified  in  the 

— s 

central  panel.  The  pattern  of  isolated  clusters  shown  is  specific  to  the  stimulus  vector  ^  .  It  is 
independent  of  the  initial  state  vector  X(0)  but  depends  as  expected  and  desired  on  the  matrix  of 
coupling  coefficients  Cy  which  was  selected  randomly  in  [0,3]  in  order  to  allow  the  appearance 

of  some  chaotic  orbits  within  the  isolated  clusters.  (Changing  the  range  of  Cy  to  [0.3-3] 

eliminates  the  chaotic  orbits  and  results  in  isolated  clusters  containing  cells  with  period-m  and/or 
fixed-point  orbits).  Other  parameters  of  the  network  were  a  =  0.1  and  €q  =  1 .  The  convergent 

pattern  of  isolated  clusters  shown  in  this  figure  is  extended  to  iteration  n=100-199  as  in  Fig.  2(a) 
which  shows  that  the  orbit  of  i=56  is  actually  intermittent  switching  between  periodic  and 
chaotic  episodes. 
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range  between  [0,4],  The  next  panel  to  the  right  of  the  //  panel  is  the  spatio-temporal 

input  vector  u(n)  with  the  values  of  ui(n)  e  [0,1]  being  color  coded  in  accordance  to  the 
color  scheme  in  the  extreme  right  panel.  The  central  large  panel  gives  the  time  evolution  or 
orbits  Xi(n)  vs.  n,  i=0,l,...N=100  of  the  100  cells  of  the  network  i.e.,  the  color  coded 
temporal  evolution  of  the  state  vector  X(n)  of  the  network.  The  first  panel  to  the  right  of 
the  central  panel  gives  the  power  spectrum  of  the  state  variable  Xi(n)  treated  as  a  phase- 
variable,  i.e.  the  power  spectrum  of  f(Xi)  =  exp(j2;rXi(n))  obtained  from  the  Fourier 
transform  of  f(Xi)  with  respect  to  i  formed  for  every  time  step  n.  This  is  shown  for  the 
last  10  iterations  of  the  central  orbits  plot  Xi(n)  window  i.e.,  for  iterations  90  to  99.  The 
normalized  power  spectrum  is  also  color  coded.  We  find  the  power  spectrum  to  be  useful 
in  two  ways:  (a)  Conceptually,  as  projection  of  the  network  activity  onto  a  different  space 
a  sort  of  perception  space,  (b)  observing  this  window  indicates  if  and  when  the  network 
converges  to  a  steady  periodic  state.  The  presence  of  disordered  or  chaotic  orbits  will 
produce  visible  perculation  in  the  power  spectrum  window.  Finally  the  four  panels  below 
the  central  window  give  the  orbits  Xi(n)  vs.  n  for  selected  cells  obtained  by  specifying  any 
4  cell  numbers  i.  In  this  fashion  the  orbit  of  an  individual  cell  within  any  cluster  can  be 
examined  quantiatively.  This  also  enables  us  to  determine  the  nature  of  an  orbit  i  e 
whether  it  is  fixed-point,  period-m,  or  chaotic  etc....  Note  the  qualitative  nature  of  the 
orbits  can  also  be  seen  in  the  central  color  coded  orbits  panel.  These  lower  plots  illustrate 
also  clearly  the  initial  transient  period  of  the  network  when  n=0  to  99  is  selected.  The 
range  of  n  covering  the  100-199,  200-299  iterations  and  so  on  can  also  be  displayed  by  the 
program  in  order  to  examine  the  longer  term  orbits  of  the  network  if  desired.  Normally  we 
find  the  network  converges  to  a  persistant  pattern  within  the  first  100  iterations  or  so  and 
quite  often  in  50  iterations. 

Most  remarkable  in  the  behavior  illustrated  in  Fig.  2  is  that  the  network  is  capable  of 
classifying  spatio-temporal  stimulus  patterns  uniquely  and  independently  of  the  initial 
condition  of  the  network  at  the  instant  of  applying  the  stimulus.  This  behavior  stems 
directly  from  the  relinquishing  of  control  over  dynamics  from  initially  extrinsic  control  to 
eventually  entirely  intrinsic  control.  The  extrinsic  stimulus  serves  to  guide  the  trajectory  of 
the  network  with  progressively  less  influence  to  a  region  of  its  state  space  where  it  is 
released  to  find  a  “convergent”  state  compatible  with  the  stimulus.  The  behavior  is  also 
remarkable  in  that  cell  orbits  of  different  type  (fixed-point,  period-m,  intermittent  and 
chaotic)  can  coexist  within  the  same  cluster.  The  handing  over  of  control  over  dynamics 
from  extnnsic  to  intrinsic,  is  also  responsible  for  the  fact  that  locations  of  active  clusters  are 

not  confined  to  cells  associated  with  high  values  of  /if  but  that  their  locations  can  cover 

cells  initially  receiving  very  low  level  of  extrinsic  activity  in  a  manner  that  is  characteristic 
of  the  stimulus  pattern. 

Another  noteworthy  aspect  of  the  behavior  of  the  network  we  have  noticed  is  that 
there  are  instances  where  cells  within  well  separated  clusters  possess  identical  synchronized 
orbits  Xi(n)  and  not  just  merely  phase-locked  orbits.  This  behavior  parallels  the 
synchronized  oscillations  of  local  field  potentials  observed  by  several  workers  in  the  brain 
of  cat  and  monkey  and  with  Eckhorn’s  modeling  of  that  behavior  in  networks  of  soikine 
neurons!  16]-[21]. 


The  independence  of  behavior  from  initial  conditions  may  be  practically  important 
because  it  suggests  that  the  network  may  be  able  to  receive  repeated  samples  of  a  time 
vaiying  stimulus  pattern  to  enhance  recognition.  An  example  from  ATR  could  be  the 
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broadband  echo  from  a  moving  aircraft  when  the  repeated  samples  are  from  the  time 
evolution  of  the  frequency  response  or  range  profile  of  the  target  as  it  changes  aspect. 

The  conceptual  similarity  of  the  isolated  clustering  behavior  in  PCLNs  and  the 
clustering  of  brain  activity  seen  in  fMRI  and  PET  raises  an  interesting  scientific  question. 
If  PCLNs  are  valid  models  of  cortical  nets  then  the  clusters  of  brain  activity  seen  in  fMRI 
and  PET  should  also  exhibit  analogous  temporal  activity.  Unfortunately  the  time  resolution 
of  fMRI  and  PET  at  present  is  two  coarse  to  discern  any  temporal  activity  within  the 
clusters  that  light-up  because  both  measure  the  change  in  blood  flow  to  active  brain 
regions.  An  increasing  number  of  studies  employing  PET  and  fMRI  show  that  different 
stimuli  or  cognitive  tasks  can  “light-up”  the  same  brain  spot.  This  strongly  suggests  a  role 
for  temporal  encoding  to  enable  differentiation.  It  would  be  interesting  to  see  if  future 
technological  advances  in  functional  brain  imaging  could  provide  the  needed  temporal 
resolution  to  answer  this  question. 

To  our  knowledge  the  remarkable  behavior  of  the  PCLN  outlined  in  this  report  has  no 
parallel  in  sigmoidal  neural  network,  coupled  map  lattices  or  cellular  automata.  Therefore 
we  believe  that  the  use  of  PCLNs  to  model  cortical  networks  and  higher-level  brain 
functions  provide  a  unique  tool  for  the  development  of  intelligent  systems  that  can  operate 
in  a  natural  environment  where  time  varying  signatures  are  the  norm  and  not  the  exception. 
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